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ABSTRACT 



A content-specific reading test is defined as a test that measures the reader’s ability 
to comprehend text in a particular subject area. To date, reading tests used to place 
students into developmental courses have been content-general rather than content- 
specific. Unfortunately, the discipline-generic model of comprehension that 
supports content-general reading tests is challenged by conclusions emerging from 
several research orientations: schema theory, construction-integration theory, 
domain-knowledge theory, and expert-novice studies. This paper briefly reviews 
how each orientation questions the efficacy of the discipline-generic model and 
proposes that content-specific rather than content-general reading tests should be 
used in developmental placement. 
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Developing Content-Specific Reading Placement Tests 
Edward H. Behrman, National University 
Presented at the 32 nd annual conference 
of the College Reading and Learning Association 
New Orleans, LA, November 6, 1999 

To date, reading assessment and reading instruction at the college level have been 
grounded largely in a discipline-generic model of comprehension. That is, there is an 
inherent assumption in many of our learning assistance efforts that reading ability crosses 
disciplinary boundaries, so that “a good reader is a good reader,” no matter the content. 
We test students using content- general tests that yield a global comprehension score and 
place students into courses based on that global score. We often advise or even require 
“weak” readers to enroll in reading improvement classes designed to instruct students in 
cross-disciplinary reading strategies that will make them better readers in all their classes. 
Yet theory and research over the past three decades have begun to erode the notion of 
comprehension as a discipline-generic ability. In fact, comprehension appears based on 
both discipline-specific and discipline-generic factors, with emerging evidence that 
discipline-specific factors may have primacy. Thus, the current emphasis on discipline- 
generic placement testing and discipline-generic reading assistance may be inconsistent 
with what most reading educators and psychologists now believe about reading. 

Each year critical placement decisions are made that affect whether students may 
enroll in credit-level or developmental courses. The construct validity of using content- 
general reading tests as part of the placement process needs to be re-examined in light of 
current views of reading and learning. AERA/APA/NCME (1985) standards for testing 
emphasize that without a solid theoretical basis for the test’s development, empirical 
evidence is meaningless. Although dissatisfied with current notions of reliability and 
validity, Schoenfeld (1999) echoes this same point when he says, “If you are going to test 
for students’ understanding of something, then (a) you have to have an adequate 
characterization of what it is you’re assessing, and (b) you need to have a good idea of 
how performance on the assessment corresponds to being able to do whatever it is that’s 
supposedly being assessed (p. 1 1).” Content-general reading tests may serve poorly as 
placement instruments because they score low in both areas: (a) they are founded on an 
assumption that reading comprehension is not mediated by the nature of the reading 
content; and (b) they attempt to predict performance in a particular course by presenting 
examinees with passages from different subjects altogether (Behrman, in press). The 
purpose of this paper is to provide theoretical support for the use of content-specific 
rather than content-general reading tests in developmental placement. Further, it will 
provide a few examples of content-specific reading tests and examine potential issues that 
may affect the development of content-specific reading tests. 

A content-specific reading test measures the reader’s ability to comprehend text in 
a particular subject area, such as history, psychology, literature, or biology. Such a test 
presents passages exclusively from the subject field being tested. The comprehension 
score indicates the examinee’s ability to understand text in that subject area. In contrast, 
a content-general reading test measures the reader’s ability to comprehend text in all 
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subject areas. Passages are drawn from a variety of subject areas. The comprehension 
score indicates the examinee’s global ability to understand text across subject areas. 
Reading tests commonly used for developmental placement are content-general. For 
example. Assessment and Placement Services for Community Colleges (APS) presents 
eight passages drawn from natural science, social science, and contemporary life (College 
Board, 1990). The Assessment of Skills for Successful Entry and Transfer (ASSET) 
presents three passages from prose fiction, business, and social science (American 
College Testing Program, 1990). The comprehension section of the Nelson-Denny 
Reading Test (NDRT) presents seven passages from humanities, natural science and 
social science (Brown, Fishco, & Hanna, 1993). Table 1 illustrates the range of topics 
included on each test. 

Unfortunately, the discipline-generic model of comprehension that supports the 
construct validity of content-general reading tests is challenged by conclusions emerging 
from several research orientations that inform our understanding of reading processes: 
schema theory, construction-integration theory, domain-knowledge theory, and expert- 
novice studies. In the subsequent sections, each research orientation will be briefly 
reviewed with attention to how each questions the efficacy of the discipline-generic 
model. 

Schema Theory 

A real estate investment adviser once told me, “The easiest way to make a million 
dollars is to start with a billion.” Similarly, the best way to comprehend text is to already 
know something about the content. Henk, Stahl, and Melnick (1993) assert that “it is now 
almost universally accepted that prior topical knowledge exerts a significant impact on 
reading comprehension and retention practices (p. 1).” Schema theory (Anderson & 
Pearson, 1984; Mason & Staff, 1984; Rumelhart, 1981) has strongly influenced reading 
educators to conclude that meaning resides not in the text alone but also in the mind of 
the reader. Readers interpret text based on their culture, experience, and content 
expertise. Ryder and Graves (1998) state, “More than any factor, the reader’s ability to 
draw upon prior knowledge will affect whether a text is comprehended and how 
efficiently it is comprehended (p. 20).” Most reading experts conceive reading as an 
interactive process in which readers gain meaning by merging text-based and schema- 
based clues. 

Readers utilize schemata to generate tentative hypotheses about text to be 
confirmed or rejected as reading continues (Mason & Staff, 1984). Anderson & Pearson 
(1984) concluded that prior knowledge improves comprehension by helping the reader 
make inferences, direct attention to important rather than trivial information, and plan for 
recall. It therefore follows that the same reader may comprehend more from a passage 
dealing with familiar content (e.g., computers) than with unfamiliar content (e.g., Russian 
literature). In addition, readers also have schemata for genre (e.g., expository), text 
organization (e.g., compare-contrast), and language use (e.g., sarcasm). But content- 
general reading assessments are insensitive to a reader’s varied content, textual, and 
language schemata. By amalgamating the examinee’s responses across the range of 
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passages, the composite score purports to represent a trait (generic reading ability) that is 
difficult to interpret in light of schema theory, which posits that comprehension is highly 
content-dependent and thus differential across content areas and text types. 

One critic of schema theory is Carver (1992a, 1992b), who proposes rauding 
theory as an alternative to better explain reading. According to rauding theory, reading 
consists of five processes: skimming, scanning, rauding, learning, and memorizing. 
Rauding is defined as being “similar to the traditional concept of general reading ability” 
(Carver, 1992a, p. 174), which proceeds at a constant rate and involves no studying. 
Carver claims that prior knowledge has little effect on rauding efficiency level, which is 
analogous to instructional reading level. Yet Carver acknowledges that prior knowledge 
is used during the learning and memorizing processes. According to Carver, a reading 
test requires learning and memorizing if (a) the test is difficult, (b) subjects have been 
instructed to read carefully or reread, or (c) subjects anticipate difficult items. Carver’s 
argument against schema theory, particularly in reference to comprehension on reading 
tests, may be moot, since most reading tests intended for admissions, placement, or 
achievement decisions present subjects with challenging material, subjects understand 
they must read carefully, and they can anticipate difficult questions. Reading tests 
therefore require to differing degrees elements of learning and memorizing, by Carver’s 
admission dependent on prior knowledge. 

Proponents of content-general testing may claim that the effects of prior 
knowledge are “washed out” by the variety of content areas presented. Such a claim is 
unfounded. In fact, composite scores from content-general tests are biased by prior 
knowledge (Johnston, 1984). Examinees are placed at a great advantage (or 
disadvantage) depending on which subject areas and topics are presented on the content- 
general test. To combat knowledge bias, Johnston has suggested pretesting examinees 
for prior knowledge and then partialling out the effects of knowledge from the 
comprehension score. This procedure would provide two scores: (a) level of prior 
knowledge, and (b) reading comprehension, independent of prior knowledge. Such a 
procedure is statistically possible, but would we want to use it for placement decisions? 
Academic reading is always rooted in specific content and academic success depends in 
part upon the reader’s ability to utilize prior knowledge. Reading achievement in the 
classroom is never “independent’ of prior knowledge. As McKenna & Robinson (1997) 
note, “Content literacy is content specific.... prior knowledge of the specific topics 
involved is a vital variable of content literacy” (p. 10). 

Construction-Integration Theory 

The construction-integration theory of reading (Kintsch, 1986, 1988; Kintsch & 
van Dijk, 1978; Marines & Kintsch, 1987; Moravcsik & Kintsch, 1993; van Dijk & 
Kintsch, 1983) also supports a discipline-specific model of reading comprehension. 
Construction-integration theory asserts that the reader engages three levels of text 
representation. Surface-level representation is explained as “processes concerned with 
the parsing of text” (Kintsch, 1986, p. 89). Words, phrases, and their linguistic relations 
are encoded into working memory. During textbase representation the reader builds 
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propositions to establish meaning of the text and works toward coherence by finding the 
relationships among propositions. Situational representation occurs as the reader 
connects the overall situation described by the text to his or her domain-knowledge 
system. Situational representation may involve adding to an existing situation model or 
developing a new one. These three processes are seen as interdependent. Textbases may 
be formed without full parse particularly when a reader faces semantic or syntactic 
difficulties. Similarly, situational representations may be developed without a well- 
organized textbase, as when the situation is well known to the reader. The textbase 
model allows the reader to recall or summarize the text and the situational model allows 
the reader to draw inferences, elaborate, and solve problems. 

Kintsch (1994) distinguishes between remembering a text and learning from text. 
Remembering text requires the reader to “reproduce it in some form, more or less 
verbatim and more or less completely, at least its gist,” while learning from text requires 
the reader to “use the information provided by the text in other ways, not just for 
reproduction” (p. 294). Remembering involves superficial understanding, but learning 
involves deeper understanding. A text may be recalled and summarized with a textbase 
model. Deeper understanding, on the other hand, depends upon a situational model. 
According to construction-integration theory, the representations needed by an examinee 
during a reading comprehension test would depend upon the nature of the test items. 
Comprehension-test items that require the examinee to reproduce or recall stated 
information would require a sufficient textbase representation. Items that require the 
examinee to expand, interpret, apply, or elaborate upon stated information would require 
a sufficient situational representation. 

In testing comprehension, we often identify comprehension items as Level 1 (on 
the line), Level 2 (between the lines), or Level 3 (beyond the lines) (see Behrman, 1998). 
Construction-integration theory implies that Level 3, sometimes called “global inference” 
or “scriptally implicit,” depends upon situational representation. Yet it is quite possible 
that a situation model may be needed for Levels 1 and 2 as well. Suppose a text passage 
on basketball states, “The point tossed an alley-oop to the center.” A comprehension 
item that asked, “What did the point do?” would appear to be a Level 1 question, as the 
answer is stated in the text. But for readers unfamiliar with basketball, even a superficial 
understanding of gist is troublesome without knowing the situational meaning of “point,” 
“alley-oop,” and “center.” Similarly, suppose a text passage on baseball states, “The 
pitcher feared walking in a run.” A comprehension item that asked, “How many men are 
on base?” would appear to be a Level 2 question, as the answer though not directly stated 
is implied by the text. Again, for readers unfamiliar with baseball, such an inference 
would be impossible. It may be that while remembering text requires a sufficient 
textbase, frequently the textbase itself requires a sufficient situation model. 

Domain-Knowledge Theory 

There has been a long-standing debate in cognitive psychology as to whether 
learning new concepts is more a function of prior domain knowledge or general 
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reasoning ability (Lawson et al., 1991). Two extensive literature reviews (Alexander & 
Judy, 1988; Byrnes, 1995) provide evidence for the efficacy of domain knowledge. 

Bymes (1995) found that in 80 percent of 33 studies from 1988 to 1994, domain- 
general ability was less important than domain-specific ability. Fourteen of these studies 
used prose memory or reading comprehension as a dependent variable. He further 
reported that although the existence of general ability (or “g” factor) is supported by 
several strands of psychological research, empirical evidence fails to identify “general 
capacity for learning” as a component of general ability. Rather, general ability depends 
upon rate of information processing, employment of strategies across tasks, and 
metacognition (planning and monitoring skills). Thus, high general-ability students tend 
to perform intellectual tasks across domains quicker, devise more effective strategies, and 
better assess their progress. Yet a domain-knowledgeable student may be able to perform 
quickly, strategize, plan, and monitor within that domain without a high level of general 
ability. Since high general ability, by definition, is uncommon, domain-specific success 
is more the rule than the exception. Bymes (1995) summarized current contemporary 
views of intellectual ability as follows: 

1 . Conceptual (declarative) knowledge is domain-specific, but procedural 
knowledge may be domain-specific or domain-general. 

2. Students differ in domain-specific concepts, procedures, and strategies. 

3. Level of prior knowledge is a good predictor of future learning. 

4. Students have uneven amounts of content-specific knowledge due to (a) 
environmental demands, including schooling; (b) topic interest; (c) rate of 
information processing; (d) preference for novelty; (e) practice; and (f) mothers 
who provide intellectual stimulation and hold high expectations. 

5. Domain-general procedures start out as domain-specific. 

Alexander & Judy’s (1988) review of research between 1975 and 1988 looked at 
the relationship between prior declarative knowledge and strategic knowledge as it 
affects academic performance. Strategies may be limited to a particular content area 
(task-limited), to two or more specific content areas (goal-limited), or generalizable 
across a wide array of content areas (general strategies). Their review, though tempered 
by methodological and conceptual issues inherent in much of the research, offered the 
following hypotheses: 

1. Accurate and complete domain-specific declarative knowledge is a necessary 
precondition for efficient use of both domain-specific and domain-general 
strategic knowledge. 

2. Accurate and complete domain-specific declarative knowledge by itself will not 
lead to successful task performance unless the learner can efficiently use strategic 
knowledge. 

3. Incorrect or incomplete domain-specific declarative knowledge may hinder task 
performance. 

4. Domain-specific declarative knowledge allows use of efficient strategies; 
conversely, lack of domain-specific declarative knowledge leads to use of 
inefficient strategies. 
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5. The relative importance of domain-specific declarative knowledge may depend 

upon the nature of the domain or the requirements of the task. 

Taken together, these two reviews suggest a construct definition of 
comprehension that is much different from that suggested by content-general tests. The 
ability to read with understanding would not appear to be constant across disciplines 
since comprehension, a critical cognitive learning activity, depends upon content-based 
prior knowledge, content-related strategies, and more generalized reading strategies. But 
some generalized strategies may be restricted to particular content areas (e.g., 
mathematics, physics, engineering). “General reading ability” is limited to ability to 
process text fluently and automatically; recognize the opportunity to use generalized 
strategies when applicable; and monitor reading progress. Further, application of reading 
strategies is enhanced by prior knowledge, so a reader who possesses “general reading 
ability” but lacks prior knowledge may still be unable to derive meaning from text. 

Expert-Novice Studies 

Overall, empirical studies that compare high-knowledge subjects (domain 
experts) to low-knowledge subjects (domain novices) support the theoretical position that 
prior knowledge is strongly related to college students’ understanding of text, although 
not all studies agree on the performance outcomes of prior knowledge. For example, 
Stahl, Hare, Sinatra, & Gregory (1991) found no differences in factual recall between 
high-knowledge and low-knowledge subjects, but high-knowledge subjects were better 
able to infer an organization of the facts. Shimoda (1993) found that topic familiarity 
improved speed and improved short-term accuracy for recognition questions. Royer, 
Carlo, Dufresne, & Mestre (1996) found that without domain expertise, a reader may be 
able to understand the gist of non-technical text, but is unable to draw inferences. 

Among college students, prior knowledge has been found to influence comprehension in 
history (Hall & Edmundson, 1992; McNamara & Kintsch, 1996; Voss & Silfies, 1996), 
literature (Zeitz, 1994), psychology (Royer, Carlo, Dufrese, & Mestre, 1996), and 
physics (Alexander & Kulikowich, 1994; Royer et al., 1996). 

Two recent studies emphasizing the important role of prior knowledge among 
college students are of particular interest. College students were pretested for history 
knowledge and then presented either expanded (well developed causal structure) or 
unexpanded (poorly developed causal structure) versions of fictitious history accounts 
(Voss & Silfies, 1996). Prior knowledge was not significantly correlated with literal 
comprehension after reading expanded text but was significantly related with 
comprehension after reading unexpanded text. In other words, prior knowledge had a 
positive effect on literal comprehension when texts were sparse in content and readers 
had to rely more on schemata. It should be noted that since researchers used fictitious 
text, subjects could not use prior knowledge of the text topic, but rather background 
knowledge of more general history concepts as well as content-related reading skills. 

To investigate how prior knowledge was affected by complexity of cognitive task, 
college students were asked to read a physics or psychology text (Royer et al., 1996). In 
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each area experts were advanced undergraduate majors and novices were students in an 
introductory class. Subjects were posttested at three increasing levels of cognitive skill 
development: (1) surface-level understanding; (2) near inference (combining information 
from two different sections of text) and far inference (combining text information with 
outside knowledge); and (3) the representational stage of problem solving in which 
subjects decided whether or not an example problem conformed to the underlying 
concept or principle in a previously stated problem. Experts outperformed novices on all 
tests, even after controlling for verbal and math SAT scores. Overall, the differential in 
performance between experts and novices increased as the level of cognitive skill 
increased: in psychology, difference increased at each level, while in physics differences 
on inference and problem-solving tests were about the same, but greater than on surface- 
level understanding. Thus, after students read technical academic text in either content 
area, expertise was significantly related to test performance, with expertise becoming 
more advantageous as tasks became more complex. Such a finding makes the proposal 
for content-specific reading assessment even more compelling, as a new generation of 
reading tests places greater emphasis on inferential thinking and problem-solving skills. 

Discussion 

Reading comprehension appears to depend on content-specific prior declarative 
knowledge, content-specific procedures and strategies, and content-general procedures 
and strategies; further, the influence of prior knowledge seems to increase as text 
sparseness and difficulty of cognitive task increase. Therefore, overall reading 
comprehension scores drawn from various content areas may have limited utility in 
academic placement decisions, since a student’s comprehension may be variable across 
content areas. Even though prior knowledge biases scores on reading comprehension 
tests, reading comprehension continues to be measured by content-general assessment. 
Any efforts to eliminate or statistically control prior knowledge as a biasing factor are 
problematic: attempting to assess comprehension independent of prior knowledge creates 
a measurement construct in conflict with the current view of the reading process as 
integration of new information with existing information. 

Content-specific reading assessment is offered as an alternative to enhance 
academic decision-making. Depending on the purpose of the assessment, content-specific 
reading assessment could focus on comprehension at the broad domain level (such as 
literature, science, or social studies); narrow domain or discipline level (such as British 
literature, geology, or ancient history); or topical level (such as Lord Byron, tectonic 
plates, or the Punic Wars). Content-specific reading assessment might contain either a 
single content-area test or a battery of separately scored and independently validated 
content-area subtests. It may include fixed-response, constructed-response, or 
performance items. Thus, a wide array of testing options is available. Significantly, 
though, what separates content-specific from content-general assessment is the inclusion 
of only text drawn from a defined content area and the absence of generalizing results 
from the defined content area to an overall comprehension score indicating generic 
reading ability. Text presented would be similar to that encountered by students in 
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academic situations and items would measure the range of comprehension activities 
expected in that content area. 

One approach to content-specific placement testing would be to develop a battery 
of reading assessments relevant to the student’s proposed course of study. For example, 
determining to place a student directly into the nursing program, or into remedial courses, 
could be based in part not on presumed “overall” reading ability, but ability to 
comprehend text in anatomy, psychology, and related subjects. In contrast to content- 
general assessment, content-specific reading assessment would measure comprehension 
in content areas to be studied. A student found deficient in the requisite content-specific 
reading ability could then receive content-specific remediation, with development of 
reading skills integrated with subject-area conceptual development. Such bridge courses 
could be taught by faculty from the academic departments specially trained in reading 
development or co-taught by faculty from the academic and developmental education 
departments. Remedial programs could thus foster higher-order inferential thinking and 
applied problem-solving in a realistic, content-specific context. 

Presently, content-specific reading tests in most academic disciplines are not yet 
commercially available. Although the ACT Assessment reports comprehension in 
Arts/Literature and Social Studies/Sciences (ACT, 1997), the combined content areas are 
too broad to be considered content-specific. The few content-specific reading tests now 
on the market are used primarily in vocational training or occupational settings. For 
example, the Ramsay Job Skills Reading Test (Form CMB) has been used to test 
applicants for mechanical repair training. The five passages are titled The Drill Press, 
Hydraulic Systems, Safety Memo, Conveyors, and Steel (Ramsay, 1996). The Ramsay 
Health Care Reading Test (Form HC-1) contains passages titled Safety With Machines, 
Care of Patients’ Belongings, Communication, CPR, and Telephone Procedures 
(Ramsay, 1992). The Revised PSB-Reading Comprehension Test (Psychological 
Services Bureau, 1993), intended for secondary and postsecondary occupational training 
programs, contains 1 1 passages, 10 related to science (see Table 2). 

As content-specific reading assessment emerges from theory into practice, 
content-specific reading tests for academic placement need to be identified or developed, 
field tested, and empirically validated. Continuing research should focus on the optimal 
level of testing (broad domain, narrow domain, or topical) and the interaction among 
prior knowledge, text coherence, and task difficulty in relation to placement decisions. 
Surely development of content-specific tests will require a tradeoff in efficiency, since 
the same reading test could not be used for all purposes, but efficiency in testing must be 
weighed against the validity of the decision-making. 



Table 1 



Topics of Reading Passages Presented on APS, ASSET, and NDRT 



APS (Form A) 


ASSET (Form C-l) 


NDRT (Form H) 


Spiders 


Prose fiction 


Homer 


Narrative essay 


Electronic mail 


Insects 


Adolescence 


Ancient Greece 


Gwendolyn Brooks 


Business ethics 




Jungian psychology 


Air pollution 




Compounds 


National monument 




Soil conservationists 


Moral breakdown 




Symbol classification 



Chief Justices 



Table 2 



Topics of Reading Passages Presented on Ramsay JSRT, Ramsay HCRT, and 

Revised PSB-RCT 



Ramsay JSRT 


Ramsay HCRT 


Revised PSB-RCT 


The Drill Press 


Safety With Machines 


Facial Features 


Hydraulic Systems 


Care of Patients ' Belongings 


pH 


Safety Memo 


Communication 


Ozone Layer 


Conveyors 


CPR 


Dr. Abbott, Surgeon 


Steel 


Telephone Procedures 


19 lh Century Women 
Health Care Workers 

Automobile Battery 

Employment Pattern 

Scientific Method 

End of the Medical 
“ Middle Ages " 



Mental Hygienists ’ 
Views of Behavior 
Problems 



Speech Clinic Study 
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